Spanning Tree Method for Minimum Communication Costs In Grouped Virtual MapReduce Cluster
نویسندگان
چکیده
Today, MapReduce and virtual cluster are sharp swords for this big data and cloud computing era. To combine these two emerging technologies, it brings feasible-scalability, easy-management, fast-deployment and high-efficiency with the system. As every sword has two sides, the I/O bottleneck of virtualization technologies may seriously impacts on the performance of MapReduce cluster which deals with I/O-intensive applications. In this paper, we analyze the combination advantages and disadvantages of virtualization technology of MapReduce cluster. We also analyze the communication model for both of them and build a communication costs model. Then, we propose a novel algorithm of minimum-weight spanning tree to construct a lower communication costs virtual MapReduce cluster. With the help of constructing minimum-weight spanning tree, we find out a method to select local-master and group the cluster. Theoretical simulation and experiment results demonstrate that our method can greatly reduce communication costs. The performance improvement is up to ~40.4% respectively
منابع مشابه
Providing a Simple Method for the Calculation of the Source and Target Reliabili- ty in a Communication Network (SAT)
The source and target reliability in SAT network is de- fined as the flawless transmission from the source node to all the other nodes. In some references, the SAT pro- cess has been followed between all the node pairs but it is very time-consuming in today’s widespread networks and involves many costs. In this article, a method has been proposed to compare the reliability in complex networks b...
متن کاملProviding a Simple Method for the Calculation of the Source and Target Reliabili- ty in a Communication Network (SAT)
The source and target reliability in SAT network is de- fined as the flawless transmission from the source node to all the other nodes. In some references, the SAT pro- cess has been followed between all the node pairs but it is very time-consuming in today’s widespread networks and involves many costs. In this article, a method has been proposed to compare the reliability in complex networks b...
متن کاملA Metaheuristic Algorithm for the Minimum Routing Cost Spanning Tree Problem
The routing cost of a spanning tree in a weighted and connected graph is defined as the total length of paths between all pairs of vertices. The objective of the minimum routing cost spanning tree problem is to find a spanning tree such that its routing cost is minimum. This is an NP-Hard problem that we present a GRASP with path-relinking metaheuristic algorithm for it. GRASP is a multi-start ...
متن کاملA Spanning Tree for Enhanced Cluster Based Routing in Wireless Sensor Network
Abstract—Wireless Sensor Network (WSN) clustering architecture enables features like network scalability, communication overhead reduction, and fault tolerance. After clustering, aggregated data is transferred to data sink and reducing unnecessary, redundant data transfer. It reduces nodes transmitting, and so saves energy consumption. Also, it allows scalability for many nodes, reduces communi...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- JDIM
دوره 11 شماره
صفحات -
تاریخ انتشار 2013